Multi-Chain Prefetching: Exploiting Memory Parallelism in Pointer-Chasing Codes

As the processor-memory performance gap continues to widen, application performance becomes increasingly limited by the memory system. Applications that employ linked data structures (LDSs) are particularly challenging from the standpoint of the memory system because of the memory serialization eeects associated with indirect memory addressing. Also known as the pointer chasing problem, such memory serialization eeects prevent latency tolerance techniques from overlapping the cache misses suuered along a chain of indirect memory references. Consequently, latency tolerance techniques are limited in their ability to tolerate the latency of cache misses arising from pointer chasing. While pointer-chasing is inherently sequential, pointer-chasing computations typically traverse multiple pointer chains in an independent fashion. Such independent pointer-chasing traversals represent a large source of memory parallelism. This paper explores the possibility of exploiting such memory par-allelism for the purposes of memory latency tolerance. First, we present a memory scheduling algorithm that computes a prefetch schedule from an LDS traversal speciication that overlaps serialized memory fetches across multiple pointer-chasing traversals. Second, we present a prefetch engine architecture capable of traversing LDSs. Our prefetch engine issues prefetches according to the prefetch schedule, thus exploiting the memory parallelism uncovered by our scheduling algorithm. Finally, we conduct an experimental evaluation of our prefetching technique on four pointer-chasing applications. Our results show multi-chain prefetching increases performance between 52% and 78%. However, early initiation of prefetches causes prefetch buuer thrashing, thus limiting further increases in performance. Additional simulations show that prefetch buuer thrashing can be eliminated in some cases by using a larger prefetch buuer and by reducing the conservative prefetch distances computed by our scheduling algorithm.